The present invention relates to an image sensing apparatus and method having a noise reduction function.
FIG. 3 is a block diagram illustrating a configuration of a field circulation type noise reduction circuit. To one of its input terminals, pixel signals output from an image sensor are inputted after being digitized by an analog-digital converter. To the other input terminal, pixel signals of the previous field outputted from the noise reduction circuit are inputted from a field memory.
The pixel signals inputted from the image sensor includes a movement component and a noise component. By subtracting the input signals from the image sensor from the signals of the previous field outputted from the field memory in the subtractor 3-1, a resultant signal component, i.e., input signal xe2x88x92(noise component+movement component), is outputted from the subtractor 3-1. Then, in the movement removal circuit 3-2, the movement component is removed from the output signal from the subtractor 3-1. The movement removal circuit 32 comprises a limitter, and does not output a signal having a frequency lower than a predetermined frequency. Accordingly, a movement component whose frequency is relatively low is removed, and only a noise component whose frequency is relatively high is outputted. Thus, only xe2x88x92(noise component) is outputted from the movement removal circuit 3-2. Then, by adding the pixel signals inputted from the image sensor and the signals outputted from the movement removal circuit 3-2 in an adder 3-3, a movement component (=(movement component+noise component)xe2x88x92(noise component)) is obtained with the noise component being removed. The signal from which the noise component is removed is stored in the field memory, and the stored signal is outputted to the subtractor 3-1 for the process in the next field.
In the aforesaid method of noise reduction processing, correlation between images sensed at consecutive timing is utilized. Therefore, if an object is still with respect to the image sensing apparatus, then corresponding points of two images sensed at consecutive different times, namely, pixel signals inputted from the image sensor to the movement removal circuit 3-2 and pixel signals inputted from the field memory to the movement removal circuit 3-2, should be expressed by the same pixel in the two images difference. In a case where the corresponding points are expressed by pixels at considerably different positions in the two images, namely, when the object moves greatly with respect to the image sensing apparatus, since there is naturally a limitation in reducing movement in the movement removal circuit 3-2, there is a possibility that effects of noise reduction will be reduced, or can not be expected at all.
Further, as a method of performing noise reduction processing in an image processing apparatus, there are a method of performing the aforesaid noise reduction processing after changing pixel signals outputted from the image sensor into luminance signals and chromatic signals in a camera signal processing circuit, and a method of performing the noise reduction processing on the pixel signals outputted from the image sensor before inputted to the camera signal processing circuit.
In the latter method, since noise reduction is performed before xcex3 processing which non-linearly increase levels of signals ranging from low frequency to middle frequency whose S/N ratio is small is applied to the pixel signals in the camera signal processing circuit. Thus, an effective noise reduction can be realized.
As a method of electrically expanding a sensed image by way of image processing, there is a method in which the sensed image is written in a frame memory or a field memory, and signals are outputted while interpolating between the signals to realize expansion. In this method, pixel signals outputted from the image sensor are converted to luminance and chromatic signals by way of camera signal processing, then expanded.
Further, there is an expansion method performed without a relatively expensive frame memory or field memory. In the method, a line memory is used for expansion in the lateral direction and expansion in the longitudinal direction is realized by controlling the image sensor. FIG. 13 is a schematic view for explaining double-sizing expansion processing, and each box corresponds to each pixel of the image sensor (area sensor). FIG. 12 shows an example of a configuration of the image sensor, and FIG. 14 is an explanatory view showing an example of filter arrangement provided on the surface of the image sensor. In FIG. 13, only an 8xc3x978 pixel array is shown to simplify the explanation, however, the image sensor usually has a great number of pixels.
In the expansion, upper one-fourth of lines (L1 and L2 in FIG. 13) is scanned fast and the scanned pixel signals are discarded. Then, the central lines corresponds to one-half of all the lines (L3 to L6 in FIG. 13) of the image sensor are scanned normally, then, the read signals are written in the line memory. If the image sensor is of a type of reading signals by two lines in a single horizontal scan period, Upon reading these central lines, they are read by two lines every other predetermined period (horizontal scan period) so as to obtain pixel signals by a set of color filters (i.e., G (green), R (red), B (blue), G in FIG. 14). The written signals are read out from the line memory during a period in which the scanning is not performed. Accordingly, each line is outputted twice, namely, from the image sensor, and then from the line memory. After the central lines (xc2xd of all the lines) are read out, the remaining one-fourth portion (L7 and L8 in FIG. 13) are scanned fast and obtained signals are discarded. In this manner, it is possible to expand the image twice larger than the original size in the longitudinal direction. A method of double-sizing the image in the longitudinal direction by reading each line twice is explained above, however, it is possible to continuously expand an image by interpolating between the read pixel signals by an arbitrary expansion ratio.
However, a problem may be posed when performing the noise reduction processing on pixel signals read from the image sensor and performing the image expansion processing in the longitudinal direction by controlling the image sensor.
When the image sensor have a color filter of a check pattern of the primary colors as shown in FIG. 14, and an expansion ratio in the longitudinal direction is continuously changed by controlling scanning of the image sensor as in the method described above, pixel signals corresponding to repetition of R and G filters may be read out as a first line of the central portion of an image to be expanded in one field, while pixel signals corresponding to repetition of G and B filters may be read out as a first line in the next field. In other words, colors of lines which are read as the first line in two consecutive fields may be different from each other.
In such a case, colors of pixel signals of the previous field will be mixed with colors of the current pixel signals and the resultant pixel signals outputted from the noise reduction circuit would be signals whose luminance and chromatic information are damaged.
Further, similar problem may be posed when performing the aforesaid noise reduction processing while performing electrical blur correction in place of the image expansion processing.
The present invention has been made in consideration of the above situation, and has as its object to obtain proper color reproduction after noise reduction in a case where electrical processing such as image expansion and blur correction are performed in an image sensing apparatus having a circulation type noise reduction processing function.
According to the present invention, the foregoing object is attained by providing an image sensing apparatus comprising: an area image sensor; a sensor operation unit which controls scanning operation of the area image sensor; area setting means for setting an area to be read of the area image sensor by controlling the sensor operation unit on the basis of an expansion ratio; a noise reduction unit which performs noise reduction processing on pixel signals obtained by consecutively scanning the area image sensor at different timings; memory which holds the pixel signals from which noise has been reduced by the noise reduction unit; a memory controller which controls reading of pixel signals from the memory in accordance with the area to be read set by the area setting means; and an expansion unit which expands an image using the pixel signals outputted from the noise reduction unit on the basis of the expansion ratio, wherein the pixel signals used in the noise reduction unit are pixel signals read from the memory and respective current pixel signals obtained by scanning the area image sensor.
According to the present invention, the foregoing object is also attained by providing an image sensing apparatus comprising: an area image sensor; a sensor operation unit which controls scanning operation of the area image sensor; a blur detector which detects blur information of a sensed image; area setting means for setting an area to be read of the area image sensor by controlling the sensor operation unit on the basis of the blur information; a noise reduction unit which performs noise reduction processing on pixel signals obtained by consecutively scanning the area image sensor at different timings; memory which holds the pixel signals from which noise has been reduced by the noise reduction unit; a memory controller which controls reading of pixel signals from the memory in accordance with the area to be read set by the area setting means; and a blur correction unit which corrects blurs of an image using the pixel signals outputted from the noise reduction unit on the basis of the blur information detected by the blur detection unit, wherein the pixel signals used in the noise reduction unit are pixel signals read from the memory and respective current pixel signals obtained by scanning the area image sensor.
Further, the foregoing object is also attained by providing an image processing apparatus for processing pixel signals obtained from an image sensing apparatus having an area image sensor and a sensor operation unit which controls scanning operation of the area image sensor, comprising: area setting means for setting an area to be read of the area image sensor by controlling the sensor operation unit on the basis of an expansion ratio; a noise reduction unit which performs noise reduction processing on pixel signals obtained by consecutively scanning the area image sensor at different timings; memory which holds the pixel signals from which noise has been reduced by the noise reduction unit; a memory controller which controls reading of pixel signals from the memory in accordance with the area to be read set by the area setting means; and an expansion unit which expands an image using the pixel signals outputted from the noise reduction unit on the basis of the expansion ratio, wherein the pixel signals used in the noise reduction unit are pixel signals read from the memory and respective current pixel signals obtained by scanning the area image sensor.
Furthermore, the foregoing object is also attained by providing an image processing apparatus for processing pixel signals obtained from an image sensing apparatus having an area image sensor, a sensor operation unit which controls scanning operation of the area image sensor, and a blur detector which detects blur information of a sensed image, comprising: area setting means for setting an area to be read of the area image sensor by controlling the sensor operation unit on the basis of the blur information; a noise reduction unit which performs noise reduction processing on pixel signals obtained by consecutively scanning the area image sensor at different timings; memory which holds the pixel signals from which noise has been reduced by the noise reduction unit; a memory controller which controls reading of pixel signals from the memory in accordance with the area to be read set by the area setting means; and a blur correction unit which corrects blurs of an image using the pixel signals outputted from the noise reduction unit on the basis of the blur information detected by the blur detection unit, wherein the pixel signals used in the noise reduction unit are pixel signals read from the memory and respective current pixel signals obtained by scanning the area image sensor.
Further, the foregoing object is also attained by providing an image sensing method in an image sensing apparatus having an area image sensor and a sensor operation unit which controls scanning operation of the area image sensor, comprising: an area setting step of setting an area to be read of the area image sensor by controlling the sensor operation unit on the basis of an expansion ratio; a scanning step of scanning the area of the area image sensor set in the area setting step; a reading step of reading pixel signals previously stored in a memory in accordance with the area set in the area setting step; a noise reduction step of performing noise reduction processing on pixel signals obtained in the scanning step using the pixel signals read in the reading step; a storage step of storing the pixel signals from which noise has been reduced in the noise reduction step; and an expansion step of expanding an image using the pixel signals from which noise has been reduced in the noise reduction step on the basis of the expansion ratio.
Further, the foregoing object is also attained by providing an image sensing method in an image sensing apparatus having an area image sensor, a sensor operation unit which controls scanning operation of the area image sensor, and a blur detector which detects blur information of a sensed image, comprising: an area setting step of setting an area to be read of the area image sensor by controlling the sensor operation unit on the basis of the blur information; a scanning step of scanning the area of the area image sensor set in the area setting step; a reading step of reading pixel signals previously stored in a memory in accordance with the area set in the area setting step; a noise reduction step of performing noise reduction processing on pixel signals obtained in the scanning step using the pixel signals read in the reading step; a storage step of storing the pixel signals from which noise has been reduced in the noise reduction step; and a blur correction step of correcting blurs of an image using the pixel signals from which noise has been reduced in the noise reduction step on the basis of the blur information.
Further, the foregoing object is also attained by providing an image processing method for processing pixel signals obtained from an image sensing apparatus having an area image sensor and a sensor operation unit which controls scanning operation of the area image sensor, comprising: an area setting step of setting an area to be read of the area image sensor by controlling the sensor operation unit on the basis of an expansion ratio; a reception step of receiving pixel signals read from the area of the area image sensor set in the area setting step; a reading step of reading pixel signals previously stored in a memory in accordance with the area set in the area setting step; a noise reduction step of performing noise reduction processing on pixel signals received in the reception step using the pixel signals read in the reading step; a storage step of storing the pixel signals from which noise has been reduced in the noise reduction step; and an expansion step of expanding an image using the pixel signals from which noise has been reduced in the noise reduction step on the basis of the expansion ratio.
Further, the foregoing object is also attained by providing an image processing method for processing pixel signals obtained from an image sensing apparatus having an area image sensor, a sensor operation unit which controls scanning operation of the area image sensor, and a blur detector which detects blur information of a sensed image, comprising: an area setting step of setting an area to be read of the area image sensor by controlling the sensor operation unit on the basis of the blur information; a reception step of receiving pixel signals read from the area of the area image sensor set in the area setting step; a reading step of reading pixel signals previously stored in a memory in accordance with the area set in the area setting step; a noise reduction step of performing noise reduction processing on pixel signals received in the reception step using the pixel signals read in the reading step; a storage step of storing the pixel signals from which noise has been reduced in the noise reduction step; and a blur correction step of correcting blurs of an image using the pixel signals from which noise has been reduced in the noise reduction step on the basis of the blur information.
Other features and advantages of the present invention will be apparent from the following description taken in conjunction with the accompanying drawings, in which like reference characters designate the same or similar parts throughout the figures thereof.